TO: POV-Ray enthusiasts everywhere (feel free to repost this)
Dear POV-Ray enthusiasts,
The POV-Team has been hard a work achieving its primary post-3.1 goal. That goal is NOTHING! <heheheh> That's right, nothing, zip, zilch, nada, null. Soon after 3.1 release I started a discussion about our plans for what's next. The #1 item on everyone's list was VACATION!
So we've taken a couple of months off development to rest, relax and enjoy the holidays (Thanksgiving through New Year's) to be with family, friends, and to actually trace a few rays. Sometimes we spend so much time working ON POV-Ray that we don't have time to play WITH POV-Ray.
We did however take the time to roughly outline our other goals and plans to achieve them. I'd like to share with you a few of the things you can look forward to in 1999 and beyond.
Bugfixes and POV-Ray 3.1b,c,d,e....
First an apology on some mixed messages regarding bugfixes. We in the POV-Team had even ourselves confused. Our Windows developer / newsgroup manger / webmaster / Chris Cason prefers his bug reports via the newsgroups where we count on other enthusiasts to help weed out the user mistakes from the real bugs. Myself (Chris Young)... I prefer email because I don't have time to monitor newsgroups. I've taken the stance that "it posted it in the newsgroup" wasn't sufficient to insure it'll get fixed.
After some discussion, the newsgroups are a better route. POV-Team members who have time to monitor the groups have assured me that they'll see to it we get all the real bug reports forwarded to me or some other appropriate team member. I'll still take emailed reports at c_young@compuserve.com if you'd like (especially bug fixes). Mac-specific bug reports are still welcome at povraymac@compuserve.com but the newsgroup is your best bet if you aren't 100% sure its a real bug.
That said... we've got several bug reports we're working on right now. I know there has been some frustration that we've not fixed them sooner and have not used your supplied fixes. We deeply appreciate the fixes, keep 'em coming. But some of the fixes are only partial fixes, some could have very serious side effects (such as changing the EPISLON value), and some have major impact on long-standing parts of the code (such as the averaged normal bug vs. the double-illuminate workaround). Nearly ALL of the recent bugs have existed since 3.0, most since 2.2 and one since before POV-Ray existed (its a bug from the old DKB-Trace upon which POV-Ray is based.)
Given our need for vacation and the age of these bugs, we've not made bug fixes a high priority but it is the next item on our agenda. Look for POV-Ray 3.1b in a few weeks.
User patches and version numbers
When we started on 3.1 it was supposed to be a "quick" release before our planned total rewrite in C++ for 4.0. Unfortunately version 3.1 took a year longer than expected. Meanwhile a number of excellent user-created patches had appeared. When 3.1 was released without these patches we were criticized for ignoring them. Many of the patches were not available until after 3.1 was well under way. Some of the patches we were not aware they existed. (Its that nasty CY-doesn't-read-newsgroups problem again <heheh>) Hey the Team Coordinator can't coordinate what he doesn't know existed can he? Anyway, I think we've recognized and corrected this problem too.
I did know about a user-submitted #macro patch and although we didn't use his code directly, it was a VERY BIG help in designing our own. As you know, #macros made it into 3.1 along with much requested arrays and file i/o which was pretty easy to include. Also halo author Zsolt Szalavari told me about his idea for generalizing all patterns such as bozo, gradient, wood etc. for use with halo. As one of the major purposes of 3.1 was to turn halo into media, it was the perfect chance to add this feature.
So version 3.1 was either based on user-created patches or some strongly demanded user features. However we have to recognize that the whole reason for having a 3.1 version was to correct problems with a haphazard integration of a user-submitted patch into 3.0. We had already implemented atmosphere when we first heard about halo. Halo was so wonderful that we could not resist putting it in even though we were nearly ready to release 3.0. While the original design looked good, it was confusing. It was inconsistent with atmosphere. It created big problems within the core code and it aggravated other long-standing design problems. Our zeal to drop in a user patch late in the game caused lots of problems and it took lots of work in 3.1 to fix them.
So we recognize that there is a BIG backlog of user patches with features we REALLY want. But do we really want to force them to wait until a major 4.0 rewrite in a new language? After much debate the answer is no.
Therefore the current plan is to create a POV-Ray 3.5 whose primary purpose is make official, as many user-submitted patches as possible. To borrow Ron Parker's term, this will be a "Super Patch" version. Note however we will not simply be rubber-stamping Ron's work in collecting patches as they currently exist. Ron's super-patch is proving useful to the team in evaluating various patches (thanks Ron) however some work must be done to integrate the new features in way that won't conflict with future plans. Also some of the features will be modified or eliminated. For example cross-platform portability is a major design priority and the iso-surface patch makes use of DLLs that are not portable. We will likely eliminate that part of the patch.
At this point we can't say what will or will not make the final cut. We'll be contacting authors of all the patches we know about. We'll be asking their permission to use their patch but WE CANNOT GUARENTEE that it will be used and you should expect that it may be modified or rewritten in some way if it does make the cut. If you have a POV-Ray 3.1 compatible patch and do not receive email from me by February 1, 1999 then I don't know about your patch and you should email me at c_young@compuserve.com to tell me about it.
We'll try to keep everyone informed of our progress as best we can.
What language to use? When? How?
As I mentioned earlier, we originally planned that our next release would be a major rewrite in C++ however we had lots of input from users and team members that a compiled Java rewrite might be a good alternative. In the 18 months since that debate, Java has not fulfilled its promise to unite the world in the peaceful harmony of a single, portable language. The whole industry (not just Microsoft) is doing things to ruin Java. Enough politics... Java is out. We're sticking with our original plan for C++.
The part we've not yet decided is when or how to convert. Some want to start with a blank page. Others want to phase-in object-oriented features. POV-Ray although written in plain C is already quite object-oriented in that OBJECT and PATTERN are abstract types that are "inherited" by more complex types that extend them. Nearly everything has constructors and destructors already defined.
The current plan is to create a design specification from a blank page to get the design right first. The team has voted almost unanimously to wait and decide later whether to do the actual implementation of that design in smaller, allegedly easier to manage chunks or to just dive in and write a new implementation from scratch. Either way, we've got design work to do first. We hope to be able to do some of that work in parallel with our work on 3.5 but that may be wishful thinking.
Add-ins, API hooks, parsers, binary formats etc.
We've had many requests for the ability to make plug-ins or DLL type capability. Although the vast majority of our users run Windows, we will not be doing anything to hurt the cross-platform portability of the program. Not all operating systems have DLL capability and even if they did, you'd have to re-compile the DLL for every platform. If you're going to do that, just patch the POV-Ray source and re-compile it. We hope that eventually as our C++ design encapsulates more of the program, it will be easier for user-developers to extend or modify our objects without messing up some other part of the program. This will be a big design goal for the new object-oriented design: make it easier to add new features.
We also get frequent requests for more API (application program interface) features in POV-Ray. The problem is that the more access that external programs have to our internal workings, the easier it is for commercial programs to blur the line between what functions are POV-Ray and what functions are not. We've made solemn promises to our developers over the years that we will protect their donated contributions to our freeware from being commercially exploited. Our tough language in POVLEGAL.DOC and our refusal to expand POV-Ray's linkability is our best effort to keep that promise.
One of the most requested internal features that outside developers want is our language parser. Although I don't know why anyone would want to convert from POV (the World's Greatest Renderer) to something inferior is a mystery to me. (<heheh> just kidding.) Seriously we do appreciate that some way to import POV scenes is a reasonable request but given our above-stated position on API or DLL links and our ban on using our source code, its a difficult request to satisfy.
One often-suggested solution is a POV binary format in which would "compile" our scene files into some allegedly more readable (by machines, not humans) format. However problems with floating-point representation and byte ordering make such solutions difficult in a cross-platform environment. It is also mistakenly assumed that loading a binary format would somehow make parsing MUCH faster. However much of the slow parsing on mega-size files is spent allocating memory for 100s of thousands of objects and textures individually. The really slow parses are slow because you're thrashing away at a virtual memory swap file. I won't take the time to re-argue the whole debate we had. I'll just say that we came very close to deciding to go ahead and implement a binary format despite the difficult technical problems and limited usefulness. As we began to discuss the details however, we soon concluded that the benefits were not worth the effort.
The POV-Ray language in most respects isn't really that hard to parse in its current form. Two issues make it complicated. POV-Ray 3.1 language isn't hard to parse but the 3.0, 2.0 and 1.0 languages are difficult to parse simultaneously. Also directives such as #if, #while, #declare, #macro etc. can appear almost anywhere and THAT is hard to deal with.
We decided on the following items which we will work on as time and manpower permit:
1) I will be writing an official technical document that describes the POV-Ray language from a parser-writer's point of view. It will include a grammar specification using the syntax notation in our reference docs. It should be helpful to anyone who wants to implement their own parser.
2) We will eventually write a separate utility or an optional feature in POV-Ray itself to translate (as much as possible) a version 1.0, 2.x or 3.0 scene into 3.1 scene and to possibly unroll loops, resolve conditionals and expand macros. This will allow you to create a much more easy-to-parse scene.
3) We will continue to look for ways to make parsing faster and easier and will look for other methods to assist utility authors needs in regards to the language.
Summary
We've got a full agenda ahead of us. We'll continue to do the best we can given our time and ability. I'm continually amazed that we have done so much already and I'm even more amazed at the wonderful art that you create with POV-Ray. It inspires us to keep working.
